University of Manitoba: description of the PIE system used for MUC-6

The PIE (Principar-driven Information Extraction) system takes a different approach to the problem o f information extraction from the NUBA system that was used in MUC-5 . The NUBA system did not have a parser and relies on an abductive reasoner to construct the semantic relationships between domain specifi c concepts mentioned in a sentence . The PIE system, on the other hand, relies heavily on a principle-base d broad-coverage parser, called PRINCIPAR [2, 6, 8], that we have developed over the past three years . Most of the information extracted are directly "read-off" the parser outputs by a subtree pattern-matcher, bypassin g the usual step of constructing semantic representations . In spite of the radical difference between the high-level approaches of our MUC-5 and MUC-6 systems , over 85% of the code of the MUC-5 system was reused . This is largely due to the unified abductive view o f different tasks in natural language understanding [7] . PRINCIPAR and the abductive semantic interprete r share the same message passing algorithm for abduction . They differ only in the contents of the messages and constraints on message propagation. The architecture of the PIE system is shown in Figure 1 . The processing is sequential . A text is firs t broken up into sentences . The lexical analyzer turns a stream of tokens into a lattice of lexical items . The lexical items may then be combined or deleted by lexical rules, which are responsible for recognizing named entities . PRINCIPAR takes the lattice of lexical items and output a dependency tree between the words in a sentence . PRINCIPAR attempts to construct a parse for the full sentence . However, when it fails to do that , it retrieves parse fragments that cover the complete sentence . Information is extracted from the dependenc y trees by a subtree pattern matcher. The format of NE and CO outputs do not meet the standard of the scoring software because of arbitrary insertion and deletion of white spaces . A separate program is used t o resolved the differences . The PIE system is implemented in about 38k lines of C++, about 33k to 34k lines were written befor e MUC-6. It contains an interpreter for LISP-like expressions so that all the knowledge structures, such a s finite automata for finding sentence boundaries, the lexicon, the lexical rules, the grammar network, an d extraction rules, are written in LISP-like expressions .